A block matching method or a correlation method based on a correlation calculation is known as a conventional method of detecting a motion vector of an image during image blur correction and the like.
In the block matching method, an input image signal is divided into a plurality of blocks of an appropriate size (for example, 8 pixels×8 lines), and a difference in pixel value between a current field (or frame) and a previous field is calculated in block units. Further, on the basis of this difference, a block of the previous field that has a high correlation to a certain block of the current field is searched for. A relative displacement between the two blocks is then set as the motion vector of the certain block.
In a method of searching for a block having a high correlation during block matching, the correlation is evaluated using a sum of squared difference SSD, which is the sum of squares of the pixel value difference, or a sum of absolute difference SAD, which is the absolute value sum of the pixel value difference. As SSD and SAD decrease, the correlation is evaluated to be higher. When a pixel position within a matching reference block region I of the current field is represented by p, a pixel position (a position corresponding to the pixel position p) within a subject block region I′ of the previous field is represented by q, and the pixel values of the pixel positions p, q are represented by Lp, Lq, respectively, SSD and SAD are respectively expressed by the following Equations (1) and (2).
                                              ⁢                              SSD            ⁡                          (                              I                ,                                  I                  ′                                            )                                =                                    ∑                                                p                  ∈                  I                                ,                                  q                  ∈                                      I                    ′                                                                                                                    ⁢                                                  ⁢                                          (                                  Lp                  -                  Lq                                )                            2                                                          (        1        )                                          SAD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                              q                ∈                                  I                  ′                                                                                                    ⁢                                                Lp              -              Lq                                                                      (        2        )            
Here, p and q are quantities having two-dimensional values. I and I′ represent two-dimensional regions of the current field and the previous field, respectively. The term pεI indicates that the coordinate p is included in the region I, and the term qεI′ indicates that the coordinate q is included in the region I′.
On the other hand, in the correlation method based on a correlation calculation, average values Ave (Lp), Ave (Lq) of the pixels pεI and qεI′ respectively included in the matching reference block region I and the subject block region I′ are calculated. A difference between the pixel value included in each block and the average value is then calculated using the following Equation (3).
                                                                                ⁢                                          Lp                ′                            =                                                Lp                  -                                      Ave                    ⁡                                          (                      Lp                      )                                                                                                                                  1                      n                                        ⁢                                                                  ∑                                                  p                          ∈                          I                                                                                                                                              ⁢                                                                        (                                                      Lp                            -                                                          Ave                              ⁡                                                              (                                Lp                                )                                                                                                              )                                                2                                                                                                                                                  p            ∈            I                          ⁢                                  ⁢                                  ⁢                              Lq            ′                    =                                                    Lq                -                                  Ave                  ⁡                                      (                    Lq                    )                                                                                                                    1                    n                                    ⁢                                                            ∑                                              q                        ∈                        I                                                                                                                                  ⁢                                                                  (                                                  Lq                          -                                                      Ave                            ⁡                                                          (                              Lq                              )                                                                                                      )                                            2                                                                                            ⁢                          ❘                              q                ∈                                  I                  ′                                                                                        (        3        )            
Next, a normalization cross-correlation NCC is calculated using Equation (4).NCC=ΣLp′Lq′  (4)
A block having a large normalization cross-correlation NCC is evaluated as having a high correlation, and the displacement between the blocks I′ and I having the highest correlation is set as the motion vector.
When an object or an image pickup subject included in an image is stationary, the motion within individual regions and the motion of the entire image match, and therefore the motion vector may be calculated by disposing the block in which the correlation calculation is to be performed in an arbitrary fixed position.
It should be noted that in certain cases, it may be impossible to obtain a highly reliable motion vector due to the effects of noise or when the block is applied to a flat portion or an edge portion having a larger structure than the block. To prevent such cases from arising, a technique for performing a reliability determination during calculation of the motion vector is disclosed in JP8-163573A and JP3164121B, for example.
Further, when the object or image pickup subject included in the image includes a plurality of motions, it is a subject to calculate the motion vector of the entire image in order to correct blur, for example In JP8-251474A, the object is divided into a plurality of regions, and an important region is selected from the plurality of regions in accordance with the magnitude of the motion vector, the size of the region, and so on. The motion vector of the selected region is then set as the motion of the entire image.
In this case, region selecting means (i) select the region having the largest area from the plurality of regions, (ii) select the region having the smallest motion vector from the plurality of regions, (iii) select the region having the largest range of overlap with a previously selected region from the plurality of regions, and (iv) select one of the region having the largest range, the region having the smallest motion vector, and the region having the largest range of overlap with the previously selected region.